Reading Paper: Swarm Learning, a machine learning method that enables peer-to-peer federated learning without explicitly identifying the central node Public.icon
\sWhen I first saw this tweet, my first question was, "Was there a Central Node for Federated Learning?" Problem\n\sConclusion: It exists Previous companies, interviews and appearances related to oneself.icon
? What makes it awesome compared to previous research?
Therefore, Swarm Learning is introduced to integrate all medical data from every data owner worldwide without violating privacy laws. Swarm Learning is a distributed machine learning approach that integrates edge computing, blockchain-based P2P networking, and coordination that maintains confidentiality without requiring a central coordinator, which can go beyond Federated Learning.
\sBig tkgshn.icon*4
In the current field of medicine and AI, "centralized AI" is primarily used. Centralized solutions have inherent disadvantages, such as increased data traffic, data ownership, confidentiality, privacy, security concerns, and the creation of data monopolies that favor data aggregators.
\s"Although Federated Learning is a solution for this, SL works better," argues this position.
? What is the key to the technology or the method?
Conceptually, machine learning can be done locally if there is enough data and computer infrastructure locally.
\s\s> https://gyazo.com/078642c36b076cac6d94e91346ea47a7
c, Federated learning. Data is stored by the data provider, and calculations are performed locally where the data is stored and available, but parameter settings are orchestrated on a central parameter server.
Is this what is meant by "Parameter Central"?
Previous companies, interviews and appearances related to oneself.icon What is "Parameter Central" in Federated Learning? Perplexity AI.icon In federated learning, a central parameter server is used to coordinate the learning process across multiple devices or clients. The central server aggregates model parameters from clients and returns updated model parameters to clients. The central parameter server plays an important role in federated learning by adjusting the learning process and ensuring that model parameters are updated correctly.
Ah, I think I understand now. I think I misunderstood.
Federated Learning clearly states that "local copies are sent back to the central server." So, it wouldn't work if the central server were destroyed.
It seems that it was still centralized. tkgshn.icon*4
Federated AI addresses some of these aspects,19,25. Data is stored locally, solving local confidentiality issues, but model parameters are still handled by a central administrator, leading to centralization of power. Furthermore, such a star-shaped architecture reduces fault tolerance.
d, The principle of SL that does not require a central custodian.
e, A schematic diagram of a swarm network consisting of swarm nodes that exchange parameters for learning, implemented using blockchain technology.
e, is this using blockchain? tkgshn.icon*3
? How was it validated as effective?
? Is there any discussion?
- It is not difficult to add smart contracts to FL, and there is already previous research.
- N magazine is heavily criticized for not understanding ML, such as unclear security benefits of changing the owner of the central node, etc.
The biggest innovation of swarm learning is that there is no central node that aggregates data, according to the post. However, the post also says that "when sharing the learning of individual models, one of the group nodes is dynamically selected as a leader, merges model parameters from each peer node collected by the selected leader node, and uses the merged parameters to start the next training batch." In my view, while swarm learning does not have an explicit central node, when the model needs to share parameters, one of the local nodes is selected as the center each time, and can be considered as an implicit central node. Also, I don't know how privacy is protected in this setting, as one local node receives all parameters from other nodes, rather than the average value of parameters, as in standard federated learning. Additionally, the authors use blockchain as a network, and state that smart contracts in blockchain prevent attacks from skeptical or dishonest participants. I am not very familiar with blockchain, so I don't understand the details of the mechanism or the benefits of blockchain. Is this a problem without the merits of blockchain?
The problem with no merit of blockchain
Looking at this paper, it is likely that it has obtained referees with backgrounds in medical data and privacy, but it is clear that it is not an ML paper. The experiments and baselines are not particularly rigorous, and overall, it is written terribly out of proportion to the technical progress, which is quite small. What a mess. tkgshn.icon*3
There is no application of blockchain here. Using blockchain for federated learning is not new, and we built one in 2018, but that was not the first time. This is just a low-quality publication in Nature.
Criticism that "It's okay for Federated Learning"
What is the innovation in removing the central node from federated learning? This can be easily achieved by performing aggregation steps with secure multiparty computations. This allows for a completely decentralized FL protocol.
I see. tkgshn.icon*4
The term "Swarm Learning" does not exist and is probably a term coined by authors who are not aware of the concepts of Swarm Intelligence or optimization. However, this is not their fault (we cannot expect people in applied science fields to be familiar with various subfields of AI and ML), and it only shows how bad it is for ML in general to easily allow such incorrect terminology.
As expected,
Previous companies, interviews and appearances related to oneself.icon was right in saying "I've heard a similar concept somewhere else before" before reading it.
If the people in that group did not know about optimization inspired by various types of nature, it would be very surprising. At least 5-10 years ago, they were quite popular in those fields (at least particle swarm optimization).
wwwwtkgshn.icon*3
Federated Learning is a somewhat misleading term due to the fact that it has a single parameter server while having a distributed worker pool.
This is true. tkgshn.icon*3
The option of having a distributed parameter server is being considered, and the commonly used names for it are "True Federated Learning" or "Peer-to-Peer Learning".
? What paper should I read next?
I read it out of interest, so there is no particular recommendation.
If I had to choose, it would be "Federated Learning" or "Swarm Intelligence: From Natural to Artificial Systems".